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We consider a stochastic model described by two stochastic differential equations 
of motion; one is for the stochastic evolution forward in time and the other for 
backward in time. We further introduce averaged quantities for the two processes and 
construct the extended thermodynamic relation following the strategy of Sekimoto 
By using this relation, we derive the fluctuation theorems such as the Seifert 
relation, the Jarzynski relation and the Komatsu-Nakagawa non-equilibrium steady 
state with respect to the introduced averaged quantities. 



I. INTRODUCTION 



In discussing non-equilibrium processes, it is sometimes important to know time evo- 
lutions not only forward in time, but also backward in time. The typical example is the 
fluctuation theorems, where several non-trivial exact relations are obtained by using the 
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different behaviors of the forward and backward processes. 

There are mainly two different approaches to discuss the fluctuation theorems; one is the 
discussion based on deterministic dynamics such as the Liouville equation, and the other is 
the stochastic approach using the Langevin equation. 

In classical deterministic dynamics, both of the forward and backward evolutions in time 
are determined by the same equation of motion by replacing t — > —t. However, we have 
to remember that the time-reversed processes in stochastic dynamics is not trivial because 
the simple manipulation, t — > —t, does not work. Let us, for example, consider the free 
Brownian motion described by 



dX(t) = v / 2/F T rfW(t), 



where v /2/F T dW(t) is a Gaussian white noise. It is known that this equation describes the 
diffusion process, where the particle is transfered from a region of higher concentration to 
one of lower concentration. Thus the inverse process must be aggregation, which cannot be 
obtained by changing t — > — t in the above equation. As a matter of fact, as we will see soon 
later, the time reversed process is described by 

dX(t) = ™cZt + y/2^dW*(t). 

That is, we need one more equation to set up a stochastic model that describes time-reversed 
processes. 

The purpose of the paper is to investigate the fluctuation theorems of this stochastic 
model. As we have mentioned, the time reversed process is important to discuss the fluctu- 
ation theorems. However, as far as we know, this non-triviality of the time-reversed process 
in stochastic dynamics has not been discussed. For this purpose, we first generalize the 
thermodynamic interpretation of stochastic dynamics following the strategy proposed by 
Sekimoto and derive the extended thermodynamic relation. Afterwards, we apply our 
stochastic model to derive the fluctuation theorems, such as the Seifert relation 2J, the 
Jarzynski relation and the Komatsu-Nakagawa non-equilibrium steady state 4]. 

This paper is organized as follows. In Sec. 2, we introduce our stochastic model. In 
Sec. 3, we develop the thermodynamic interpretation of the stochastic dynamics following 



Ref. The various fluctuation theorems are discussed in Sec. 4. Section 5 is devoted to 
the concluding remarks. 
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II. TIME REVERSED PROCESS IN STOCHASTIC DYNAMICS 

We consider a stochastic process X(t) described by the following stochastic difference 
equation, 

7dX(t) = u(X(t),a,t)dt+ ^/2f3- 1 dW(t) (dt>0), (1) 

where u is a force term, dX.(t) = ~K(t + dt) — X(£), and the noise term satisfies the following 
properties, 

(dW(t)) = 0, (dWi(t)<FNj(t)) = 5ij\dt\. (2) 

Here a represents an external control parameter such as an external field. The parameters 7 
and j3 characterize the magnitude of the relaxation and the noise, respectively. We consider 
that (3 is constant. In the following calculation, the parameter 7 does not play any important 
role, and, for the sake of simplicity, we use the special unit of 7 = 1. 

To discuss the fluctuation theorems, we need to define the time reversed process of this 
stochastic time evolution. In the classical deterministic dynamics where the noise term 
dW(t) vanishes, the time-reversed process is described by replacing t with —t in Eq. (CD). 
However, the time reversed process of the stochastic dynamics is not obtained by this simple 
replacement because of the noise term dW(t). 

For this purpose, we introduce one more stochastic difference equation which describes 
the stochastic evolution backward in time, 

dX(t) = u*(X(t),a,t)dt+ ^/2f3~ 1 dW*{t) (dt<0), (3) 

where 

(dW*(t)) = 0, (dW*(t)dW*(t)) = Sijldt], (4) 

and cfW and dW* do not have any correlation. Because dt is negative, this equation describes 
the backward process. 

From the stochastic equations ([T]) and ([3]), we obtain the two Fokker-Planck equations 

d t p(r,a,t) = -V- (u(r,a,t)p(r,t)) +/?" 1 V 2 p(r,a,t), (5) 
d t p{r,a,t) = -V>*(r,a,t)p(r,t))-/rVV(r,a,t). (6) 
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When Eq. ([3]) describes the time-reversed process of Eq. (TO , the two Fokker-Planck equa- 
tions should be equivalent, and consequently, u and u* are not independent anymore. As a 
matter of fact, the two equations are rewritten as 

d t p(r, a, t) = V ■ (p(r, a, t)v(r, a, *)) = V ■ J(r, a, t), (7) 
V ■ (p(r, a, t)b(r, a, t)) = f^^r, a, t), (8) 



where 



v(r,a,t) = -{u(r,a,t) + u*(r,a,t)}, (9) 
b(r,a,t) = -{u(r, a,t) - u*(r,a,t)}. (10) 

The solution of the second equation is 

, / x ^1,-,, / x c(a,t) V x A(r, a, t) 

b r,a,t = /T Vlnp r,a,t + 1 ; + 1 ' ; . 11 

p[r,a,t) p(r,a,t) 

Here c(a,t) and A(r, a, t) are an arbitrary scalar and vector functions, respectively. We set 
c(a, t) = and V x A(r, a, t) = for the sake of simplicity. Then u* is related to u by the 
following relation, 

u*(r,a,t) = u(r,a,t) - 2/3 _1 Vlnp(r, a, *). (12) 

It should be emphasized that, as we will see soon later, this relationship is the origin of the 
fluctuation theorems. 

In short, the forward and backward stochastic equations of motion are given by 

dX(t) = u{X(t),a,t)dt+ ^/2p- 1 dW{t) (dt>0), (13) 

dX(t) = [u(X(t), a, t) - 2(3~ l V lnp(X(t), a, t)\dt + v /2/F T cfW*(t) (dt < 0), (14) 

respectively. 

We apply this stochastic model to, for example, the free Brownian motion, where the 
forward stochastic equation of motion is 

dX(t) = ^2/3~ 1 dW(t) (dt > 0). (15) 

When we assume p(r, 0) = 5(r), the solution of the Fokker-Planck equation (|SJ) is given by 

-r 2 /(4/3-ii) 
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On the other hand, the backward stochastic equation of motion is given by 

dX(t) = ^Q-dt + y/2p- 1 dW*(t) (dt < 0). (17) 

The force term — gives rise to aggregation overcoming diffusion due to the noise 
\j2/3~ l dW*(t). We checked numerically that the above equation describes the time-reversed 
process of the free Brownian motion (|T5l) . 



III. EXTENDED THERMODYNAMIC RELATION 

In order to discuss the fluctuation theorems, it is necessary to know the thermodynamic 
interpretation in our stochastic model. 

The thermodynamic interpretation of the stochastic dynamics without time-reversed pro- 
cesses was proposed by Sekimoto [l|. When the stochastic dynamics is given by Eq. (JT]), 
Sekimoto proposed the following relation, 



dU(X(t), a) = SQ{X(t), a) + SW(X(t), a) 



where U is the energy of a Brownian particle defined by 



u(r, a, t) = —VU{r,a,t), 



(18) 



(19) 



and 



5Q(X(t),a) = VU(X{t),a)odX{t), 
5W(X(t),a) = d a U(X(t),a)oda. 



(20) 
(21) 



Here, the product o means the Stratonovich definition of the stochastic product 5j. The 
equation (I18p is obtained by the stochastic differentiation along the stochastic trajectory 
{X(t)}. In this sense, this equation is an identity Sekimoto assigned the thermodynamic 
interpretation for the r.h.s. of the identity; 5Q is the heat which a Brownian particle receives 
from the environment, and 5W is the work which is done to a Brownian particle from the 
environment. This thermodynamic relation has been studied from various points of view 



6] 



However, it is not straightforward to apply this interpretation to our stochastic model. 
Note that U does not have explicit time dependence in Sekimoto's definition. However, in 
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FIG. 1: The forces acts on a Brownian particle at t. We set dt > 0. 

our case, the energy of a Brownian particle in the backward process U* which is defined by 
u*(r, a, t) = — VC/*(r, a, t), has an explicit time dependence because of the time dependence 
of p (See Eq. ( TT2"j) ). To apply Sekimoto's procedure to our stochastic model, we introduce 
"averaged" thermodynamic quantities and discuss the thermodynamic relation for these 
quantities. 

Note that the force which acts on a Brownian particle at t is given by u(X(i), t)dt for the 
forward process and — u*(X(t), t)dt for the backward process, as is shown in Fig. [T] (dt > 0). 
Then we can introduce an averaged force which acts on a Brownian particle at t as 

/*c r / \ , ,^ r/ , . , u(X(t), a, t) — u*(X(t), a, t) , . . 

u(X(t), a, t)dt = b(X(t), a, t)dt = v 2 — dt - ( 22 ) 

Correspondingly, the averaged energy is defined as 

u(X(t), a, t) = -Vt7(X(t), a, t), t7(X(t), a, t) = U ( X (t),a,t) - U*(X(t),a,t) ^ ^ 

Following the discussion by Sekimoto, we implement the stochastic differentiation of U 
by using the Ito formula and obtain the thermodynamic relation for this averaged thermo- 
dynamic quantities as 



dU(X(t), a, t) = 5Q(X(t), a, t) + SW(X(t), a, t) + SQj(X(t), a, t), (24) 
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where 

sqm), a, t) = ™m),a,t)-vu-m),a,t) o rfX(()i (25) 

MWt), a, f) = WU')-WUO ^ (26) 

<5Qj(X(t),a,t) = -/3 -1 9 t lnp(X(t),a,t)dt 

= _ / g-i e ^(x(t),a,t) v . j(x(t), a ,t)dt. (27) 

Note that X(t) is a stochastic variable, we have to use the Ito formula instead of the Taylor 

IT 

expansion [5|]. In the derivation of the third equation, we used the macroscopic current 
defined from the Fokker-Planck equation as, 

3tp(r,a,t)=V-J(r,a,t). (28) 



One can see that the first two terms on the r.h.s. of Eq. ( |24j) already appear in Sekimoto's 
thermodynamics relation, and 5Q and 5\V are interpreted as the averaged heat and the 
averaged work, respectively. However, the last term does not appear in Eq. ( TT81) . 

In order to clarify the meaning of the new term 5Qj, let us consider the one-dimensional 
free Brownian motion. Then SQj is calculated as 

It is known that the expectation value of the position of a Brownian particle is given by 
(X 2 (t)) = 2/3~H, and one can easily see that 5Qj vanishes in this case. Thus this term should 
be interpreted as another heat which a Brownian particle receives from the environment; 
when X(t) moves slower than the average, SQj is positive and a Brownian particle recieves 
heat, and vice versa. Apparently from Eq. (1271) . this heat appears only when p has an 
explicit time dependence, that is, there is a non- vanishing macroscopic flow J. 

Note that, for the free Brownian motion, the energy of the forward process U does not 
exist, but the averaged energy U is still finite. By using Eqs. (fT2|) and (|T6j) . the averaged 
energy is calculated as 

U(X(t),t) = + ^ ln(4^- 1 t). (30) 



Then the extended thermodynamic relation f[24]) for the free Brownian motion is explicitly 
given by 

x 2 (t) , /r 1 \ /r 1 x*(t)\ x(t)odx(t) 

ln(4vr/3 t) \ = [—--— \ dt + . (31) 



At 2 v '/ V 2t At 2 2t 



s 



The last term on the r.h.s. comes from SQ, and there is no contribution from 5VV in this 
case. 

It should be noted that 5Qj is re-expressed as, 

5Qj(X(t) t a,t) = -/3V(e /3p(x{t) ' a ' i) J(X(t),a,t))rft 

-J(X(t), a, t) ■ b(X(t), a, ^eP^^dt. (32) 

The first term on the r.h.s. is a total differentiation and may vanish for the total amount 
of the thermodynamic quantities. Note that b defined in Eq. (II 01) is the averaged force on 
a Brownian particle and the second term is expressed as the product of the macroscopic 
current J and the microscopic force b. Remember that in the irreversible thermodynamics, 
the entropy production a is assumed to be given by the product of an irreversible current 
Ji rr and the corresponding thermodynamic force F, 

a = 3 irr ■ F. (33) 



One can see that the form of the second term of Eq. (132]) is very similar to this structure. 
In a quasi-static process, we may be possible to identify 

5Q = TdS, (34) 
SW = -PdV. (35) 

Then the extended thermodynamic relation is rewritten as 

dU = 5Qj + TdS - PdV. (36) 

This is very similar to the generalized thermodynamic relation discussed in the extended 
irreversible thermodynamics 

IV. FLUCTUATION THEOREM IN STOCHASTIC PROCESSES 



In this section, we will discuss the various fluctuation theorems in our stochastic model. 
Note that the fluctuation theorems based on the stochastic dynamics have already been 
discussed in several works {2, 8]. Differently from these works, our fluctuation theorems are 
expressed in terms of the averaged thermodynamic quantities introduced in the previous 
section. 
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Now we consider a set of a stochastic trajectory {X(£)} which is described by Eq. (JTJ). 



Then, by ca 
Ito formula 



culating explicitly the total differentiation of U along this trajectory using the 
||, we find the following relation, 



dU(X(t),a,t) = (dU(X(t),a,t) - dU*(X(t),a,t))/2 = -/5~ 1 c/lnp(X(t), a, t), (37) 
where 

dlnp(X(t), a, t) = [dt + u(X(t), a, t) V + /? _1 V 2 ] lnp(X(t), a, t)dt + <9 a lnp(X(t), a, t) o da 
+ v /2^ 1 Vlnp(X(t),a,t) ■ rfW(t). (38) 

Here the product in the inner product of the last term obeys the Ito definition 
Thus, for the evolution from i to tf, we have 



p(X(t f ),a f ,t 



11 = e -P(U{X(tf),a f ,t f )-U{X(t ),a ,to)). (39) 



p(X(t ),a ,t c 

A. Quasi-static change between two different stationary states 

For the quasi-static process where we assume d t p(r,a,t) ~ 0, the extended thermody- 
namic relation is approximately given by 

dU(X(t), a, t) « 5Q(X(t), a, t) + <JW(X(t), a, t). (40) 

Thus, from Eq. (1391) . the two stationary state are connected by the following relation, 

Psta(X(tf), a f ) _ e _^ E /- o i(5Q( X (t),o,t)+«VV(X(t),a,t)) - ( 41 ) 



p sta (X(t ),a ) 

Clearly, there is no contribution from the macroscopic flow, SQj, in this case. 

B. Seifert relation 

We consider a stochastic trajectory T = {X ,Xi,--- ,Xf} which is a solution of the 
stochastic difference equations (JT]), and introduce the corresponding time-reversed process 
denoted by F'. Then we consider the conditional probability which is defined by 

(42) 
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where p is the solution of the Fokker-Planck equation (J5J). Here we used that the process is 
Markovian. From Eq. fl39|) . the r.h.s. of the above equation is given by 

P (Xj, a h tj) = p(Xi, Oi, t.) e -/5(&(X J ,a,^)-f/(X l ,a l ,t ! )) = e -^pc,,a Jl t,)_ (43) 

The transition probability from an initial X to a final Xf along the stochastic trajectory F 
is, finally, given by 

-P(r(X , a , t ; X/, a/, tf)) 

= P(Xf, cif, t/|X/_i, a/_i, t/_i)P(Xj_ 1; a/_i, t/_i|Xj_ 2 , a/-2, 2) ' ■ • P(X 1 , a 1; ti|X , a , to) 

= e -/3[EL ^( x -^A)-c/(Xo,ao,io)]_ (44) 

Similarly, the conditional probability for the inverse process is 

p(r t (x ,a ,t ;X / ,a / ,t / )) 

— P(X Q , a , t \X 1 , ai, ti)P(X 1; ai, ti|X 2 , a 2 , t 2 ) • ■ ■ P(Xf-i, a /-i) a/, tj) 

= e -^[Ef = o^( X i.»»*i)-^(X/,a/,t/)]_ (45) 

Following the discussion by Seifert j^j], we introduce P(r(X , ao, to! Xf, af,tf)) as follows, 

-R(r(x , a , to; X/, a/, £/)) 

j n P(r(X a , , to; X/, a/, t/))p(X , a , to) 



■P(r t (Xo,a ,to;X / ,a/,t/))p(X / ,a / ,t 



= i n P( X o^o,to) _ j3{ p {x t ) _ u(X ,a ,t )) 
p(X f ,a f ,tf) 

= In ^°' a °'*°) _ £ V(5Qj(X,, a,, i 4 ) + 5Q(X t , a,, i 4 ) + 5W(X U a t , U)). (46) 
p(X f ,a f ,tf) 

By using this, we can derive the exact relation, 

= J rf 3 X rf 3 X / (P(r(X , ao, t ; X f , a f , t f ))p(X , a Q , fo ) e -«(r(Xo,a ,toiX / ,a /> t / ))^ 

= 1, (47) 

were ( )w means the average over possible trajectories fixing X at t and Xf at tf. Here 
we used the following relations, 

p(X ,a ,t ) = J d 3 Xf(P(T^X ,a ,t ]Xf,a f ,tf))p(Xf,af,tf)) w , (48) 
p(X f ,a f ,tf) = J d 3 Xo(P(T{Xo,a ,to]X f ,af,tf))p(Xo,ao,to)) w . (49) 
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This is the main result, leading to various fluctuation theorems. 

The expression ( |4"Tj) corresponds to the relation discussed by Seifert |2j, but the expression 
of R is different. Seifert gives the following representation for R, 

= In gggl^) + /3 1> ° ^ = In P(Xo,ao,to) _ g 5g(Xj; 
p(X f ,a f ,tf) P (X f ,a f ,t f ) ^ 

Thus our exact relation (]47|) is not equivalent to the Seifert relation. 

The validity of the Seifert relation was numerically checked {{J. However, our result is also 
consistent with this numerical result. In {{J, the stationary state with fixed a is considered. 
In this limited situation, our averaged quantities are reduced to 

SQj = 5W = 0, 5Q = 5Q. (51) 

Then our R in becomes equivalent to -R SC ifcrt- In this sense, our result is still consistent 



with 



9]. 



C. Jarzynski relation 

It is easy to derive the relation corresponds to the Jarzynski relation |3j from our relation 
(|47|) . We consider that the evolution of a Brownian particle from one stationary state 
p s ta{ r , Oo)j to the other one Pst a (r, a/). The stationary state is given by 

Psta (r,a) = -\-e^ u ^\ (52) 

where Z sta (a) = J d^re~ l3U ^ r ' a \ Substituting it into Eq. (1471) . we obtain the Jarzynski 
relation of our stochastic model, 

where 

AF = -r 1 l n m^4. (54) 
D. Komatsu-Nakagawa non-equilibrium steady state 



Recently, the general expression of non-equilibrium steady state was derived in |4|, 1 10 ] . 
The similar expression can be obtained in our stochastic model. 
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From Eq. (|46j) . we have 

P(r f (X , a , t ; X f , a f , t f ))p(X f , a f , tf) 
= P(r(Xo, ao, t ; X f , a f ,tf))p(X , a , t ) e - Wo^^x^,*,)^ (55) 

Similarly, for the inverse process, we obtain 

-P(r(X , a , t ; Xf, af, tf))p(X , a , t ) 
= P(rt(Xo, a , t ; X /; a f ,tf))p(X f , a f , tf ) e -^H^o,ao,to^ f ,a f ,t f ))_ (5g) 

By combining these equations, we have 

P(r(X , a , to! X/, a f ,tf))p(X , a , to ) e -^o,a ,t ;X f ,a f ,t f ))/2 

= P(rt(X , a , t ; X f , a f ,t f ))p(X f , a f , t f ) e -B& f*. (57) 

This is re-expressed as 

MX at)- MX a t ) W ' Xf ' " f ' tf ))e-W<*><"*>*"'-''»'*) w 

p(X f , a„ t t ) - p(X„, to ) (i3(rt(Xo _ - - — - i/))e _ R(rt(Xo .„ JolX/ .„„„ ))/2)iv (58) 

Note that Xf and X are fixed parameters and independent of the average ( )w This is 
essentially the same as the expression derived in 4] (Eq- (10)), except for the definition of 
R. The quantity Qj in Q], which corresponds to R in our case, coincides with Eq. fj46|) when 
5Qj vanishes. That is, the Komatsu-Nakagawa non-equilibrium steady state is realized in 
our stochastic model only when there is no macroscopic flow. 



V. CONCLUDING REMARKS 



In this paper, we considered the stochastic model incorporating the forward process and 



the backward process in time. Following the strategy of Sekimoto [lj, we constructed the 
thermodynamic relation for the "averaged" quantities of the forward and backward processes. 
Our thermodynamic relation is extended so that a term 5Qj associated with a macroscopic 
flow appears. We further discussed the fluctuation theorems in our stochastic model and 
derived the new expressions with respect to the averaged quantities. 

The new term SQj can be expressed as the product of a macroscopic current and a force. 
This is the form expected from the extended irreversible thermodynamics. This term may 



have a relation to the excess heat discussed in 
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The consideration of the forward and backward stochastic equations is, in fact, well known 
in Nelson's formulation of quantum mechanics, which is one of the hidden variable theories 

n 

|12|]. As a matter of fact, we can introduce a kind of a wave function even in the classical 
Brownian motion and it is possible to interpret the fluctuation theorems in terms of the 
phase of the wave function. This will be reported in another work. 

In this work, we discussed the stochastic equation of motion without inertial terms under 
a Gaussian white noise. It is also interesting to extend our discussion to the equations with 
inertial terms, memory effects and colored noise. In such a situation, it may be possible 
to investigate the dynamics of the pre-equilibrium state as is discussed in 13j. Moreover, 
if our discussion is applicable to relativistic Brownian motion Jjj], we can investigate the 
thermodynamic relation in relativistic systems. 
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